Mobile devices, including for example mobile phones (so-called “smart” phones), global positioning system (GPS) devices, and even laptop/netbook computers, provide diverse remote data capabilities. Many mobile applications are based on so-called “cloud” services, such as location services, messaging services, and so forth. Currently most cloud services are based on statically prepared information, and do not offer real-time analytics of dynamically captured events. For example, a weather report or traffic report may be updated periodically to a local news website.
Users of mobile devices, however, are coming to expect more frequent updates of events, often in real-time. For example, users may not just want a statically prepared traffic report (e.g., delays on the interstate due to an earlier accident). In addition, more real-time information is becoming readily available (e.g., concerning the location, speed, and accident involvement of many individual cars). Of course, to most users the interesting and more useful information is not the individual data (such as each car's position and speed), but information that has been analyzed and summarized for the user in real-time (e.g., the average or moving average speed of a lane of traffic on the interstate). This information can be derived from the individual data using various analytics.
The information is typically analyzed using a continuously running query. The continuous query executes cycle-by-cycle to process the data stream on a chunk-by-chunk basis. The query commits results to the data structure on a cycle-by-cycle basis to make the chunk-wise results visible to users. But in the case that the information is derived from a large (or “infinite”) number of events, the information is also infinite. Accordingly, the data structure storing the information continues to grow and can quickly become too big to handle by most mobile devices.
Mobile devices, such as those running WebOS and HTML 5, may download data in batch to ease the bandwidth limitations. However, conventional batching involves data copying and moving, which still incurs performance overhead and, in the case of very large data structures, may still cause service interruptions.
Archiving has also been used for data warehouse management applications. In these archiving techniques, the latest data is maintained in the table and older data is archived from time to time. But archiving the older data also involves moving and copying the data from the table. For example, the data may be moved from a first table holding the latest data, to another table, so that the first table only holds the new data. While this approach has been used for handling slowly-updated data in data warehousing applications, archiving is not efficient for supporting real-time analytics.